Model-based Reinforcement Learning: A Survey

Sequential decision making, commonly formalized as Markov Decision Process (MDP) optimization, is an important challenge in artificial intelligence. Two key approaches to this problem are reinforcement learning (RL) and planning. This monograph surveys integration of both fields, better known model-based learning. Model-based RL has two main steps: dynamics model planning-learning integration. In comprehensive survey the topic, authors first cover learning, including challenges such dealing with stochasticity, uncertainty, partial observability, temporal abstraction. They then present a systematic categorization integration, aspects as: where start planning, what budgets allocate planning real data collection, how plan, integrate acting loop. conclusion discuss implicit end-to-end alternative for potential benefits RL. Along way, draw connections several related hierarchical transfer contains broad conceptual overview combination optimization. It provides clear complete introduction topic students researchers alike.